Invertible Concept-based Explanations for CNN Models with Non-negative Concept Activation Vectors

Abstract

Convolutional neural network (CNN) models for computer vision are powerful but lack explainability in their most basic form. This deficiency remains a key challenge when applying CNNs in important domains. Recent work on explanations through feature importance of approximate linear models has moved from input-level features (pixels or segments) to features from mid-layer feature maps in the form of concept activation vectors (CAVs). CAVs contain concept-level information and could be learned via clustering. In this work, we rethink the ACE algorithm of Ghorbani et al., proposing an alternative invertible concept-based explanation (ICE) framework to overcome its shortcomings. Based on the req..